Omnidirectional camera for use in police car event recording

ABSTRACT

A system and method for an omnidirectional camera for use in recording events around a police vehicle is disclosed. The system and method include an omnidirectional camera and a digital processor for processing the omnidirectional images captured by the omnidirectional camera. The digital processor may be operable to locate one or more regions of interests disposed within the omnidirectional images. A recordable medium is also disclosed for storing at least some of the captured images.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to and incorporates by reference theentire disclosure of U.S. Provisional Patent Application No. 61/024,328filed on Jan. 29, 2008.

TECHNICAL FIELD

In general, this application relates to video-recording devices and moreparticularly, but not by way of limitation, to omnidirectionalvideo-recording devices for use with law-enforcement vehicles.

BACKGROUND

Cameras and other video-recording devices have long been used to capturestill and video images. In general, cameras consist of an enclosedhollow portion with an opening or aperture at one end to allow light toenter and a recording surface for capturing the light at the other end.In addition, cameras often have a lens positioned in front of theaperture along an optical axis to gather the incoming light and focusall or part of an image onto the recording surface. Fields of view varyfrom camera to camera, but in general, most cameras have a field of viewthat ranges from a few degrees to, at most, 180°.

In the past, to overcome the limited field of view, surveillance camerasused for monitoring large areas were oftentimes mounted to mechanismsadapted to enable the camera to pan, tilt, and zoom in order to moveobjects into the camera's field of view. One type of camera, called anomnidirectional camera, has been used to monitor large areas without aneed for mechanisms to enable pan, tilt, and zoom. An omnidirectionalcamera is a camera with an omnidirectional field of view, such as, forexample, a 360-degree field of view. Some omnidirectional cameras may beadapted to capture images from all directions (a full sphere). However,many omnidirectional cameras do not capture a full sphere of images, butrather capture 360 degree of images along a single axis with the fieldof view being limited angularly above and below the axis.

The use of dashboard cameras in police vehicles has been well known formany years and is an integral part of a police department'sevidence-gathering capability. One limitation of conventional cameras isthe limited field of vision. Devices that include a movable camera andhaving near 360-degree capability have been developed. One limitation ofthese devices is the time it takes to pan or tilt the camera. Anadditional limitation relates to the reliability issues commonlyassociated with devices having moving parts. More recently, devices withat or near 360 degree image-capturing capability have been developedthat do not require mechanical panning, tilting, and zooming. However,these devices often require large amounts of data storage and oftenrecord large amounts of irrelevant images.

SUMMARY

In view of the foregoing and other considerations, the present inventionrelates generally to video-recording devices and more particularly, butnot by way of limitation, to omnidirectional video-recording devices foruse with law-enforcement vehicles.

In accordance with one aspect of the present invention, a system isprovided for capturing and storing images, the system including anomnidirectional camera mounted to a vehicle and operable to capture anomnidirectional image of a scene surrounding the omnidirectional camera;a digital processor coupled to the omnidirectional camera and operableto receive the captured omnidirectional image; the digital processorbeing operable to locate one or more regions of interest within theomnidirectional image; and a storage medium coupled to the digitalprocessor and operable to receive and store a first subset of theomnidirectional image corresponding to the one or more regions ofinterest.

More specifically, the system may also include wherein the digitalprocessor is operable to compress the first subset to a first resolutionand to compress a second subset of the omnidirectional image to a secondresolution; the second subset of the omnidirectional image is stored inthe storage medium at the second resolution; and wherein the firstresolution is greater than the second resolution. The system may alsoinclude wherein the digital processor is operable to delete theomnidirectional image other than the first subset. The system may alsoinclude a wireless microphone disposed within the scene; and wherein thedigital processor is operable to utilize a signal-detection algorithm todetermine a location of at least one of the one or more regions ofinterest based at least in part on one or more signals received from thewireless microphone. The system may also include wherein the digitalprocessor is operable to utilize a gaze-estimation algorithm todetermine a location of at least one of the one or more regions ofinterest based at least in part on the direction a person is looking.The system may also include wherein the digital processor is operable toutilize an object-detection algorithm to determine a location of atleast one of the one or more regions of interest. The system may alsoinclude an optical target disposed in the scene; and wherein the digitalprocessor is operable to utilize an optical-target detection algorithmto determine a location of at least one of the one or more regions ofinterest.

In accordance with another aspect of the present invention, a method isprovided for capturing and storing images, the method includingproviding an omnidirectional camera mounted to a vehicle and operable tocapture an omnidirectional image of a scene surrounding theomnidirectional camera; transmitting the omnidirectional image to adigital processor coupled to the omnidirectional camera; locating, viathe digital processor, at least one region of interest within theomnidirectional image, the at least one region of interest correspondingto a first subset of the omnidirectional image; compressing the firstsubset to a first resolution; and storing the compressed first subset ina storage medium coupled to the digital processor.

More specifically, the method may also include compressing a secondsubset of the omnidirectional image to a second resolution; wherein thefirst resolution is greater than the second resolution; and storing thesecond subset of the omnidirectional image in the storage medium. Themethod may also include deleting a second subset of the omnidirectionalimage the second subset being mutually exclusive of the first subset ofthe omnidirectional image. The method may also include wherein the atleast one region of interest corresponds to an area of the sceneimmediately surrounding a law enforcement officer. The method may alsoinclude coupling an antenna to the digital processor; detecting, via theantenna and the digital processor, a signal from a wireless device;determining, by the digital processor, from what direction the signalcame; and using the determined direction to locate the at least oneregion of interest. The method may also include disposing an opticaltarget in the scene; detecting, via the digital processor, a location ofthe optical target; and determining the at least one region of interestvia the detected location. The method may also include estimating adirection in which a person is looking; and determining the at least oneregion of interest via the estimated direction.

BRIEF DESCRIPTION OF THE DRAWINGS

A more complete understanding of the method and apparatus of the presentinvention may be obtained by reference to the following DetailedDescription when taken in conjunction with the accompanying Drawingswherein:

FIG. 1 is an image-capturing system;

FIG. 2 is a field of view of the image-capturing system of FIG. 1;

FIG. 3 is a side view of a field of view of the image-capturing systemrelative to a police vehicle;

FIG. 4 is a perspective view of region of interest located in a field ofview of the image-capturing system relative to a police vehicle; and

FIG. 5 is a flow chart of a method for capturing omnidirectional images.

DETAILED DESCRIPTION

FIG. 1 shows an image-capturing system 10. In the embodiment shown, thesystem 10 includes an omnidirectional camera 12 coupled to a computer16. The omnidirectional camera 12 may include a camera 11 arrangedadjacent to an external lens 13 and a dome 14, the dome 14 being concaverelative to the camera 11. The dome 14 and lens 13 in combination areadapted to allow light to pass therethrough. In some embodiments, thedome 14 may be convex relative to the camera 11, the dome 14 and lens 13in combination being adapted to reflect light towards the camera 11. Thecamera 11, in combination with the dome 14 and the lens 13, may formpart or all of the omnidirectional camera 12. The omnidirectional camera12 may be adapted to capture a single omnidirectional still image and/ormay be a video camera adapted to sequentially capture a plurality ofomnidirectional images. The omnidirectional image may be a 360-degreeimage of a scene surrounding the omnidirectional camera 12, wherein 360degrees is relative to an optical axis 22 of the camera 11. As referredto herein, an omnidirectional camera is a camera adapted to captureomnidirectional images where the omnidirectional camera may be anycamera and/or camera system adapted to capture wide-angle images from awide-angle field of view up to and including 360-degree images from a360-degree field of view. As referred to herein, an omnidirectionalimage is an image captured by an omnidirectional camera where theomnidirectional image may be a wide-angle image from a wide-angle fieldof view up to and including a 360-degree image from a 360-degree fieldof view. In some embodiments, the omnidirectional camera may have afield of view ranging from on the order of 180°, 190°, 200°, 210°, 220°,230°, 240°, 250°, 260°, 270°, 280°, 290°, 300°, 310°, 320°, 330°, 340°,350°, and/or 360° and the omnidirectional images may be less than orequal to the omnidirectional camera fields of view. In some embodiments,the omnidirectional camera 12 may be a high-definition camera such as,for example, a camera having a sensor adapted to capture images on theorder of several Megapixels.

The lens 13 may be adapted to focus omnidirectional images, such as awide-angle lens, a super-wide-angle lens, a fish-eye lens, a full-circlelens, a spherical mirror-type lens, a conical mirror-type lens, or otherlens and/or mirror configuration capable of focusing omnidirectionalimages. In some embodiments, the computer 16 may be a standalone unitand/or may be remotely disposed from the omnidirectional camera 12, butin the embodiment shown is integrated with the omnidirectional camera12. The computer 16 typically includes a digital processor coupled to adata-storage device 18 that may be used to store at least a portion ofcaptured images. The data-storage device 18 may include, for example, aninternal hard drive, an external hard drive, and/or awritable/rewritable drive, such as a CD and/or DVD drive.

FIG. 2 shows a field of view (FOV) of an embodiment of anomnidirectional camera 12. For descriptive purposes, a coordinate systemhas been overlaid having an optical axis 22 shown running verticallyalong the optical axis of the omnidirectional camera 12 and a horizontalaxis 23 perpendicular thereto and passing through the lens 13. Ingeneral, the FOV of a camera is the area of a scene around the camerathat can be captured by the camera. The FOV 21 of the omnidirectionalcamera 12 along the horizontal axis 23 is shown. The FOV 21 extends bothabove and below the horizontal axis 23. For example, in the embodimentshown, the FOV 21 extends approximately 10 degrees above the horizontalaxis 23 and approximately 45 degrees below the horizontal axis 23. Invarious embodiments, the FOV 21 may extend more than or less than 10degrees above the horizontal axis 23 and/or may extend more than or lessthan 45 degrees below the horizontal axis 23. Although FIG. 2 shows theFOV 21 along one axis, the full FOV of the omnidirectional camera 12 mayinclude all 360 degrees of rotation about the optical axis 22. Theentire panorama of the omnidirectional camera 12 would then be a55°×360° FOV, where the 55 degrees represents the size of the anglerelative to the horizontal axis 23.

Referring now to FIG. 3, an omnidirectional camera 12 mounted on theceiling of a police vehicle 31 is shown. It can be seen that the FOV 21of the omnidirectional camera 12 extends outwardly from the policevehicle 31 so that images of objects from the surroundings of thevehicle 31 that are within the FOV 21 can be captured. In the embodimentshown, the omnidirectional camera 12 is mounted to the interior of thepolice vehicle 31. However, the omnidirectional camera 12 may be adaptedto be mounted in a variety of other locations, such as, for example, ona dashboard, on a rearview mirror, on an exterior roof, on or near atrunk, and/or on or near a hood of the police vehicle. Similarly, theomnidirectional camera 12 may be adapted to be mounted on a car,motorcycle, boat, helicopter, van, truck, and/or any mobile orstationary location where monitoring a surrounding would be desirable.

In some embodiments, the omnidirectional image may be a high-resolutionimage and may be sent to a digital processor to be analyzed, compressed,and/or stored. Oftentimes, the high-resolution omnidirectional image maybe compressed before storage to reduce the amount of memory needed tostore the omnidirectional image. Because the omnidirectional camera maycapture images from less than or an entire 360 degrees, large portionsof the omnidirectional image being captured may be irrelevant. In someembodiments, the digital processor may separate the omnidirectionalimage into subsets and compress the subsets to different resolutionsbefore storing some or all of the subsets. For example, subsetsdetermined to be more relevant may be stored at a higher resolution thansubsets determined to be less relevant. In some embodiments, lessrelevant subsets may be stored at a very low resolution or may bediscarded instead of being stored so that data-storage capacity of thedata-storage device is not consumed by the less relevant subsets. Insome embodiments, the subsets of the omnidirectional image may be largeregions, such as quadrants, and only those subdivisions determined to berelevant are stored or are stored at a higher resolution than the othersubdivisions.

Referring now to FIG. 4, an omnidirectional camera 12 is shown mountedto an external roof of a police vehicle 31. In the embodiment shown, theomnidirectional camera 12 captures an omnidirectional image of the scenesurrounding the police vehicle 31 within the FOV 21. In the embodimentshown, an area of the scene containing a person has been located as anarea of interest within the FOV 21 of the omnidirectional camera 12. Aregion of interest (ROI) 41 may then be defined within theomnidirectional image corresponding to the located area of interest. Forexample, a digital processor (not shown) may be adapted to define theROI 41 to include the subset of the omnidirectional image immediatelysurrounding a police officer. In some embodiments, the image capturedmay be a panoramic image of less than the full 360 degrees surroundingthe camera, where the digital processor defines the ROI 41 to includeless than the entire panoramic image.

In some embodiments, the digital processor may be adapted to track anobject, such as a person, as the location of the object in the FOV 21changes by moving the ROI 41 correspondingly. As will be described inmore detail below, the digital processor may be adapted to utilize oneor more detecting and/or tracking algorithms to determine where tolocate and/or move the ROI 41, such as, for example, a signal-detectionalgorithm for tracking a signal of a wireless microphone worn by theofficer, a gaze-estimation algorithm for estimating a direction a personis looking, an object-detection algorithm for identifying and trackingspecific features of an object, a target, or a person, amotion-detection algorithm for identifying movement of objects, and/oran algorithm for allowing user input. In some embodiments, the ROI 41may be stored at a relatively higher resolution while the remainingareas of the captured omnidirectional image may either be discarded orstored at a lower resolution. In some embodiments, an entireomnidirectional image may be discarded if it is determined that no ROIis present at that particular time.

The above-mentioned signal-detection algorithm may include one or moreantennae coupled to the digital processor for determining a location ofan officer relative to the camera. For example, as an officer walks froma driver's door of the police vehicle around a front of the policevehicle, signals originating from a signal-generating device such as,for example, a wireless microphone worn by the officer, will reflect themovement. The digital processor may be adapted to define the ROI 41 asthe subset of the omnidirectional image from the same direction as theorigination of the signals from the signal-generating device. Forexample, when the officer is standing next to the driver's door, the ROImay be a front-left quadrant relative to the police vehicle of theomnidirectional image. When the police officer moves around to thepassenger side, the ROI may be a front-right quadrant relative to thepolice vehicle of the omnidirectional image. In various embodiments, thesubset containing the ROI 41 may be more or less than a quadrant of theomnidirectional image.

The above-mentioned gaze-estimation algorithm may be utilized toestimate which direction a person within the FOV is looking. An ROI maythen be defined as the subset of the omnidirectional image from thatdirection. When the omnidirectional camera is mounted inside a policevehicle, the omnidirectional image captured may include areas from boththe interior and the exterior of the police vehicle. In someembodiments, the digital processor may be adapted to determine theorientation of a person's head and estimate the direction the person islooking.

In some embodiments, a portion of the omnidirectional image beingcaptured may include a facial region of a person, for example, a driveror passenger of a vehicle. In some embodiments, the digital processormay be adapted to determine the direction a person is looking byanalyzing the direction a person's eyes are pointing. The ROI can thenbe defined as the subset of the omnidirectional image in that direction.In some embodiments, the gaze-estimation algorithm may be calibrated foraccuracy by having a driver look at several reference points during acalibration process. In other embodiments, the gaze-estimation algorithmmay automatically detect the direction without requiring a calibrationprocess.

In some embodiments, the accuracy of the gaze estimation may allow anarea where a person is looking to be pinpointed to within approximately5° to 7°. In some embodiments, accuracy may be improved by tracking aperson's eye movements as the person views the edges of an object. Themovements may then be compared to objects in the FOV in the directionthe person is looking. For example, if a person is looking at a spheresitting next to a cube, the person's eyes will make more roundedmovements rather than straight movements along an edge of a cube. Thedigital processor may be adapted to detect this difference and definethe ROI as the sphere, rather than the cube. In some embodiments, theobject may then be tracked even after the person looks away. In someembodiments, the object is no longer tracked once the person looks away.

The above-mentioned object-detection algorithm may include variousalgorithms adapted to detect various features of an object of interestin order to identify and track the object. For example, an opticaltarget may be disposed on an officer and an optical-target detectionalgorithm may be utilized to track the officer. In some embodiments, theoptical target may be a part of the officer's uniform, such as forexample, a badge or cap of the officer. In some embodiments, the opticaltarget is an object specifically designed to facilitate tracking of theofficer. In other embodiments, a facial-feature tracking algorithm maybe adapted to locate human faces within the omnidirectional image. AnROI may then be defined to include the located face. In someembodiments, a facial-recognition algorithm may be utilized to identifythe person in the FOV.

In some embodiments, an object-outline algorithm may be utilized todetect a person in an image by detecting outlines of various bodyportions. For example, an outline of a head, while difficult todifferentiate from other round objects, may be used to detect thepresence of a person in the FOV if the outline of shoulders is alsodetected in a head-and-shoulders type relationship. In some embodiments,a vehicle-detection algorithm may be utilized to detect the presence ofvehicles within the FOV. For example, reference points may be taken fromvarious points around an object to determine if the object is a vehicle.In some embodiments, reference points may be taken from around thevehicle to determine the size and shape of the vehicle and to identifythe make and model of the vehicle. In some embodiments, a still image ofthe ROI may be saved, the ROI may be saved at a higher resolution thanother areas of the image, and/or information about the ROI may be savedas metadata. In various embodiments, the object-detection algorithm maybe operable to automatically detect the presence of one or more of aplurality of objects such as, for example, a license plate, a body part,such as a head, face, or limb, a weapon, a flash of a weapon discharge,and/or any other object that may be desirable to detect and/or track. Insome embodiments, the algorithm may be adapted to automatically detectsome or all of a plurality of objects and/or the algorithm may beadapted to allow a user to select one or more of a plurality of objectsfor the algorithm to detect and/or track.

In the above mentioned motion-detection algorithm, movement of an objectwithin the FOV may be detected and an ROI may be defined to track themoving object. For example, the algorithm may be adapted to locateobjects that exhibit known motion patterns, such as, for example, ahuman gait, a moving vehicle, such as an approaching or recedingvehicle, a moving person, sudden motion changes, such as a car accident,predetermined gestures by a person in the FOV, and/or other detectablemotions. In some embodiments, the sensitivity of the algorithm may beadjusted by a user so that minor or irrelevant movements will nottrigger the creation of an ROI. For example, in some embodiments, thealgorithm may be adapted to reject an ROI or possible ROI based onspatial and/or motion analysis, such as, for example, cars passing anofficer during a traffic stop.

In the above-mentioned algorithm for allowing user input, a digitalprocessor may be adapted to define an ROI based at least in part oninput received from a user. For example, a user control may be coupledto the digital processor for allowing a user to designate the ROI, suchas, for example, a joystick, a mouse, a trackball, a directional buttonsuch as a pan/tilt/zoom button or buttons. In some embodiments, acaptured image is displayed on a viewable screen or projector and anarea is outlined and/or highlighted on the display. The user may movethe area and/or move the image being displayed to define the ROI.

Referring now to FIG. 5, a flow chart of a process 500 for capturing anomnidirectional image is shown. In the process 500, an image-capturedevice, such as an omnidirectional camera, captures an omnidirectionalimage in a field of view of the camera at step 502. In some embodiments,the camera may be mounted relative to a police car and adapted tocapture an image while the police vehicle is moving and also when thepolice vehicle is stopped, for example, during a traffic stop.

From step 502, execution proceeds to step 504. The capturedomnidirectional images are sent to a processor at step 504. At step 506,one or more regions of interest (ROI) in the captured image are located.In some embodiments, the raw data of the captured image may be read andan automatic ROI locator algorithm may be run. In some embodiments, thedigital processor may, for example, run a facial-feature locationalgorithm to identify whether there are people in the field of view. Invarious embodiments, one or more location algorithms are run on raw datacoming from the omnidirectional camera. In some embodiments, one or moreof the algorithms may be run on compressed data and a feedback signalsent as to the location of the ROI.

After the one or more ROI have been located, at step 508, the digitalprocessor uses the location information relative to each of the ROIs tocompress each of the ROIs to a first resolution. For example, thelocation information may be one or more sets of coordinates and/orvectors. At step 510, non-ROI subsets of the image are compressed to asecond resolution. At step 112, some or all of the compressed image isstored on a recordable medium such as, for example, a DVD.

Although various embodiments of the method and apparatus of the presentinvention have been illustrated in the accompanying Drawings anddescribed in the foregoing Detailed Description, it will be understoodthat the invention is not limited to the embodiments disclosed, but iscapable of numerous rearrangements, modifications and substitutionswithout departing from the spirit of the invention as set forth herein.

1. A system for capturing and storing images, the system comprising: anomnidirectional camera mounted to a vehicle and operable to capture anomnidirectional image of a scene in a field of view of theomnidirectional camera; a digital processor coupled to theomnidirectional camera and operable to receive the capturedomnidirectional image; the digital processor being operable to locate anarea of interest from the scene and define a first subset of theomnidirectional image corresponding to the area of interest as a regionof interest and a second subset of the omnidirectional image mutuallyexclusive of the first subset; a data-storage device coupled to thedigital processor and operable to receive and store the first subset ata first resolution; and the digital processor being operable to performat least one of the following: discard the second subset; and save thesecond subset at a second resolution lower than the first resolution. 2.The system of claim 1, wherein the field of view of the omnidirectionalcamera is a 360-degree field of view.
 3. The system of claim 1, whereinthe field of view of the omnidirectional camera is less than 360degrees.
 4. The system of claim 1, comprising: a signal-generatingdevice disposed within the scene; and wherein the digital processor isoperable to utilize a signal-detection algorithm to define the region ofinterest based at least in part on one or more signals received from thesignal-generating device.
 5. The system of claim 4, wherein thesignal-generating device comprises a wireless microphone.
 6. The systemof claim 1, wherein the digital processor is operable to utilize agaze-estimation algorithm to define the region of interest based atleast in part on a direction a person is looking.
 7. The system of claim1, wherein the digital processor is operable to utilize anobject-detection algorithm to define the region of interest.
 8. Thesystem of claim 1, comprising: an optical target disposed in the scene;and wherein the digital processor is operable to utilize anoptical-target detection algorithm to define the region of interest. 9.The system of claim 1, wherein the digital processor defines the regionof interest based in at least partial dependence on a user input.
 10. Amethod of capturing and storing images, the method comprising: providingan omnidirectional camera mounted to a vehicle and operable to capturean omnidirectional image of a scene in a field of view of theomnidirectional camera; transmitting the omnidirectional image to adigital processor coupled to the omnidirectional camera; locating, viathe digital processor, an area of interest from the scene; defining, viathe digital processor, at least one region of interest within theomnidirectional image corresponding to the area of interest, the atleast one region of interest corresponding to a first subset of theomnidirectional image; storing the first subset at a first resolution ina data-storage device coupled to the digital processor; and performing astep selected from the group consisting of: storing a second subset ofthe omnidirectional image at a second resolution in the data-storagedevice, the first resolution being greater than the second resolution;and discarding the second subset of the omnidirectional image.
 11. Themethod of claim 10, wherein the field of view of the omnidirectionalcamera is a 360-degree field of view.
 12. The method of claim 10,wherein the field of view of the omnidirectional camera is less than 360degrees.
 13. The method of claim 10, wherein the second subset ismutually exclusive of the first subset.
 14. The method of claim 10,wherein the at least one region of interest corresponds to an area ofthe scene immediately surrounding a law enforcement officer.
 15. Themethod of claim 10, comprising: coupling one or more antennae to thedigital processor; detecting, via at least one of the one or moreantennae and the digital processor, a signal from a wireless device;determining, by the digital processor, from what direction the signalcame; and using the determined direction to define the at least oneregion of interest.
 16. The method of claim 10, comprising: detecting,via the digital processor, a location of an object disposed in the fieldof view of the omnidirectional camera; and defining the at least oneregion of interest via the detected location.
 17. The method of claim16, wherein the location of the object is detected in at least partialdependence on a pattern of motion of the object.
 18. The method of claim10, comprising: estimating a direction in which a person is looking; anddefining the at least one region of interest via the estimateddirection.
 19. The method of claim 10, wherein the at least one regionof interest is defined based in at least partial dependence on userinput.
 20. A method of capturing and storing images, the methodcomprising: providing a 360-degree camera mounted inside a policevehicle and operable to capture a 360-degree image of a scene in a fieldof view of the 360-degree camera; transmitting the 360-degree image to adigital processor coupled to the 360-degree camera; locating, via thedigital processor, an area of the scene containing an object ofinterest; automatically defining a region of interest within the360-degree image corresponding to the area of the scene containing theobject of interest; compressing to a first resolution a first subset ofthe omnidirectional image corresponding to the region of interest;compressing to a second resolution a second subset of theomnidirectional image, the second subset being mutually exclusive to thefirst subset and the second resolution being lower than the firstresolution; and storing the first subset and the second subset in astorage medium coupled to the digital processor.